Radio spectrum is scarce and the Federal Communications Commission (“FCC”), Department of Defense (“DoD”) and other international spectrum management organizations are constantly looking for ways to more efficiently utilize this limited spectrum. Demand for spectrum is continuing to rise due to the explosive growth of data, voice, messaging, and video applications. One solution to meeting the need for improved spectral efficiency as measured by bits/Hz/user is adaptive radios (also referred to as dynamic spectrum access (DSA) or cognitive radios (CR)). Adaptive radios can change their transmission characteristics to maximize transmission capacity and coverage while conserving spectral usage. Because CR represents a rich technical research area, as well as a potentially significant commercial and military products market opportunity, large numbers of research and development entities from academic, commercial and government organizations are participating in activities towards producing CR devices and systems.
One of the challenges of deploying adaptive radio technology is that it cannot be fielded without comprehensive behavior evaluation, and it cannot be evaluated in a densely populated, live environment for fear of potentially interfering with existing spectrum users (primary users). Field evaluation is preferable to lab evaluation but requires a realistic environment where it can be verified that the System under Test (SUT) will not interfere with primary users or other spectrum users. Laboratory evaluation is more cost and schedule effective, repeatable, controllable and observable, but generally lacks in realism, especially with respect to RF environmental considerations including the interactive effects of other devices/systems operating in the RF environment. These interactive effects can be between devices/systems that are adaptive/cognitive and operate in different spectrum bands, but dynamically change their frequencies in response to spectrum conditions. Addressing this phenomena has not been a part of prior art testing approaches.
There is an established and growing need to comprehensively evaluate behavior of these new adaptive devices/systems in known and postulated environments which include other representative RF systems to establish behavior characteristics. Traditional evaluation methods are increasingly stressed by the proliferation and diversity of the devices/systems and operating environments. Historically, device/system evaluation has fallen into two broad categories, field evaluation and laboratory simulation/evaluation. Field evaluation as illustrated in FIG. 1 involves placing some number of devices in a realistic field environment and exercising them to evaluate performance against specified functionality. Full-featured field evaluations place the wireless transceivers in a field scenario containing some representative RF environment where they will be operated while evaluation data is collected. These sorts of evaluations are often expensive and complex to orchestrate, and can lack flexibility since mixes of test transceiver numbers/types/locations, incumbent RF user numbers/types/locations and RF propagation conditions cannot be systematically varied to collect comprehensive data. FIG. 1 schematically depicts a typical field evaluation equipment setup. Wireless Transceiver Units Under Test (UUT) 100 operate in some RF environment 110. The RF emissions are subject to the noise, path loss, multipath transmission and interferers found in the local RF environment 110. Test instrumentation 120 is established to measure the performance of the UUT and other primary users (PU) of the RF environment. In order to accomplish a field evaluation of this variety, the UUT 100 must be physically located in the evaluation RF environment 110, and test instrumentation 120 must be constructed. In order to vary the numbers/types/locations of UUT and PU, physical units must be acquired and placed in the RF environment. In order to vary the RF environment, different field venues must be available. Additionally, test instrumentation must be provided and adapted for each UUT/PU/test environment scenario where testing is to be accomplished.
Many factors must be considered when selecting and configuring the field evaluation area including the specific type and host platform for the SUT, the characteristics and quantity of other RF devices and interferers in the environment, and environmental factors that affect the radio propagation including terrain and morphology. Field evaluation methods have been viewed as the most realistic, but many growing challenges limit their ability to be compelling. These challenges include:                Difficulty and complexity in evaluating high platform dynamic systems        More devices/systems to evaluate        More functionality & complexity to exercise including adaptive/cognitive behavior        Evaluation ranges require a broad set of realistic physical layouts        Requirements to emulate location-specific RF environments including propagation and interferers        Requirements for conditions not realizable on evaluation ranges including prohibition by FCC rules        RF environment control difficult due to encroachment of commercial RF sources.        
All of the above lead to increased costs, longer schedules, more requirements on field evaluation assets and ranges, and potentially lower confidence in results.
For adaptive RF systems, field evaluation is not practical. Laboratory evaluation methods are generally more cost and schedule effective, are more controllable and observable, but generally are lacking in realism, especially with respect to RF environmental considerations including other RF sources.
There exist many variations of lab evaluation approaches, but they can be generally bounded by “RF Path Simulator” and “Software Modeling” variants. The RF Path Simulator approach shown in FIG. 2, which interconnects RF systems/devices with conventional laboratory test equipment such as signal/noise generators, is only applicable to simple RF environments, small numbers of devices/systems under test with simple antenna systems, and small number of primary users/interferers. Lab-based evaluation using cable-based interconnection for RF emissions of UUT and the RF environment is a prior art approach to testing to overcome the challenges of placing and monitoring devices in the field environment. FIG. 2 depicts a typical lab-based equipment setup. As in field evaluations, Wireless Transceiver Units Under Test (UUT) 100 are acquired and instrumented with Test Instrumentation 120. Instead of the RF environment being that found in the field, RF test equipment such as signal generators are used to produce Interferers 210, Noise Generators 220, and Path Simulators 200 to simulate path loss and multipath in an RF channel. RF Interconnection 230 is accomplished using RF cables such as coaxial cables. This test set up approach reduces some of the complexities of field evaluation, but introduces new concerns over RF environment realism. Further, it still requires the physical introduction of new UUT and RF test equipment into the configuration for comprehensive transceiver configuration and RF environment results.
Traditional methods that use software modeling approaches as shown in FIG. 3 have historically made simplifications about the physical environment/radio propagation effects, and generally cannot support any hardware in the loop (HITL) test cases. Their validity is therefore limited to a narrow group of test cases and not well suited to the adaptive RF system evaluation problem. A variation on RF cable-connected lab testing has become more prevalent and straightforward as wireless transceiver devices have tended towards digital waveforms and digital hardware or software implementation. FIG. 3 depicts a typical framework for modern wireless communications devices as defined by the prior art OSI model. Here, different functions in the Wireless Transceiver 100 are allocated to layers in the functional stack 300. The physical layer in stack 300 is where the waveform-related functionality is contained. The physical layer can be segregated into a digital implementation portion 310 and an analog portion 320. Typical functions in the digital transmit portion 310 are waveform generation 330 and digital to analog conversion 340. Typical functions found in the analog portion 320 are baseband to RF conversion 350. Other digital processing functions associated with non-physical layers (2 through 7) are performed through digital data processing blocks 360.
A laboratory-based evaluation approach that combines the advantages of true RF path/environment emulation and HITL, but implemented in the digital domain under software control, has the potential to deliver the advantages of the different lab methods with the realism of field testing. The test platform disclosed in commonly owned U.S. Pat. No. 8,521,092, titled “Wireless Transceiver Test Bed System and Method”, which is hereby incorporated by reference, follows this approach. The present disclosure adds improvements directed to a system and method for providing a laboratory-based, field realistic, virtual RF environment where RF systems (communications, radar, jammers, etc.) produced by unassociated third parties can participate in an interactive gaming environment to evaluate behavior. This facet of the test bed problem is further described below.
FIG. 4 of U.S. Pat. No. 8,521,092, titled “Wireless Transceiver Test Bed System and Method” is included as FIG. 4 in the current disclosure to describe the operation of one embodiment. FIG. 4 illustrates the virtual wireless channel (VWC) 400 and test instrumentation plane (TIP) and metadata manager 410. The TIP may also be embodied as a database. The UUT physical layer digital portion is connected to the VWC 400 via interconnections 420, as is the TIP via 425. The VWC function is to provide a realistic wireless channel model including noise, interference, UUT signal path loss and UUT signal multipath transmission. The VWC 400 can be configured with a selectable number of virtual spectrum users (VSU) and other interferers to accurately simulate the RF environment that might be encountered in different parts of the world. The VSU may have selectable interactivity parameters, including transmission parameters and kinetics, or physical characteristics. For example, transmission parameters may include frequency, bandwidth, power, modulation. Physical characteristics, or kinetics, may include location, speed, direction of motion, and antenna parameters including type, elevation gain, azimuth gain, phase, polarization and orientation. The VSU can be selected to be a transmitter only, a receiver only or a transceiver. The VSU can be selected to be a communication device, a sensor such as a radar, a navigation device, or a jammer and can be the same type or different than the UUT. The VWC also allows for selecting transmission parameters and physical characteristics of the physical UUT.
A key feature of the VWC is that it accepts and passes analog RF or digitized RF to and from the UUT. In this way, the full effects of the wireless channel can be included in the simulation. The TIP 410 acts as a control mechanism to orchestrate the sequencing of the test bed simulation, and to collect instrumentation data at the RF and other OSI layers of the UUT. A key part of the TIP is the metadata manager. Metadata is defined as data that must be passed between the VWC and the UUT to allow real time parameters to be modeled and analyzed. As an example, metadata can include the relative locations of the UUT and VSU in a geographic region. As the simulation progresses, the delay characteristics of the multipath and relative time of arrival of the signals at each node can be accurately modeled.
Perhaps the most challenging part of adaptive RF system behavior evaluation is addressing the interaction between RF systems (including adaptive RF systems) in the field. An anticipatable adaptive RF system behavior pattern (“system 1”) may be that it adapts in response to another RF system (“system 2”) in the field (like changing RF frequency of operation), which causes system 1 to adapt (like lowering its transmission pattern), which causes system 2 to adapt (by changing RF frequency back to its original center frequency), and so on. These conditions are not producible in the field or laboratory today, in part, because many of the adaptive RF systems that will be in the field in the future do not exist today in either a “test equipment” form or “prototype form” to facilitate behavior evaluation. In fact, many future adaptive devices are only available as laboratory R&D models in university, commercial and government R&D facilities.
Based on a review of the available RF system test beds that exist in industry and academia (including those referenced in U.S. Pat. No. 8,521,092), a wireless transceiver test bed approach, capable of allowing unassociated third parties to participate in interactive spectrum gaming environments to evaluate behavior is not known.
The present disclosure utilizes emerging technologies and trends in the areas of computer networking, digital signal processing, wireless device design, wideband networks, computer and software architecture/capability and software-based modeling to provide a means to address these shortcomings. Specific technology innovations that contribute to various aspects of the present disclosure include:                digital signal processing power and available algorithms and models        ability to digitize RF with high fidelity        emerging software defined radio (SDR) software architectures, such as SCA (Software Communications Architecture)        emerging commercial off-the-shelf digital radio and SDR components (hardware and software)        ever increasing broadband connectivity between distributed sites        comprehensive and advanced RF propagation models        RF emitter models being built in software        proliferation of radio functionality being digital and implemented in software with discrete events (bits, bursts, frames, etc.).        standardization of baseband digitized interfaces to SDRs (such as the VITA-49 Radio Transport Protocol).        
The present disclosure is not limited to adaptive wireless devices in the application area of communications, but broadly applies to all wireless devices and networks including receive only, transmit only and diverse applications such as sensing, radar, and jamming. Further, it is not limited to behavior evaluation of adaptive RF systems and could also be used to evaluate conventional RF systems.
In summary, a large number of organizations are involved in the development of adaptive RF systems including industry, academia, and government. Methods, tools, and metrics to collaboratively and comparatively judge the behavior of these systems (either individually or interactively) do not exist. Progress in maturing the designs for cognitive RF systems, understanding their performance, and introducing them into the field are hampered by the lack of behavior evaluation capabilities. The disclosed system provides a means to enable the behavior evaluation in a cost effective and engaging way.